A multimodal-input multimedia-output guidance system: MMGS
نویسندگان
چکیده
We have built a multimodal-input multimedia-output guidance system called MMGS. The input of a user can be a combination of speech and hand-written gestures. The system, on the other hand, outputs a response that combines speech, three-dimensional graphics, and/or other information. This system can interact cooperatively with the user by resolving ellipses/anaphora and various ambiguities such as those caused by speech recognition errors. It is currently implemented on a SGI workstation and achieves nearly real-time processing.
منابع مشابه
Natural Language Dialogue System for Information Retrieval
The objective of our work is the development of a natural language dialogue system for information retrieval with multimodal input and multimedia output. Overall, the system consists of three phases: input analysis, information and knowledge management and output generation. The dialogue system is designed for consulting old Mexican historical documents. In this paper we describe the designed a...
متن کاملDesign of a rich multimodal interface for mobile spoken route guidance
We present a design of a rich multimodal interface for mobile route guidance. The application provides public transport information in Finland, including support for pedestrian guidance when the user is changing between the means of transportation. The range of input and output modalities include speech synthesis, speech recognition, a fisheye GUI, haptics, contextual text input, physical brows...
متن کاملMultimodal Interfaces
Multimodal systems process two or more combined user input modes— such as speech, pen, touch, manual gestures, gaze, and head and body movements— in a coordinated manner with multimedia system output. This class of systems represents a new direction for computing, and a paradigm shift away from conventional WIMP interfaces. Since the appearance of Bolt’s (1980) “Put That There” demonstration sy...
متن کاملChapter to appear in Handbook of Human-Computer Interaction, (ed
Multimodal systems process two or more combined user input modes— such as speech, pen, touch, manual gestures, gaze, and head and body movements— in a coordinated manner with multimedia system output. This class of systems represents a new direction for computing, and a paradigm shift away from conventional WIMP interfaces. Since the appearance of Bolt’s (1980) “Put That There” demonstration sy...
متن کاملIntegrating Modality Theory in Educational Multimedia Design
Multimedia technology is increasingly being used as an alternative way of delivering instruction. According to researchers, the success of multimedia is due to the dualcoding aspect of the information processing theory. In applying dual-coding principles, different information has to be coded in different media in order for people to learn effectively. Designers of interactive multimedia applic...
متن کامل